reference or affidavit under 37 C.F.R. § 107(b) to support this conclusion or withdraw the rejection. 

III. REMARKS 

The Examiner and the Applicant agreed in the telephone interview of 15 July 1997 that: 

1. The Applicant would amend claims 1, 94, 106, and 1 14 replacing "display area" with "display 
screen". This Amendment accomplishes that change; and 

2. Claims 1, 94, 106, and 1 14 as amended above overcome the Baker and Bronson references. 

Claims 109 and 134 are amended to delete an unnecessary adjective "first". 

Claims 142, 160, and 161 are amended to correct typographical errors. 

Claims 147 and 155 are amended to add the word "either", making clear that each claim 
includes either a confiner or an activation area. 

Claims 104 and 152 are amended to better correspond with the terminology used in their 
respective parent claims. 

The Examiner, during the in person interview of 10 July 1997, recalled having seen a pertinent 
reference. Immediately after the interview, the Examiner found this reference, the Bronson patent 
cited herein, and provided Applicant with a copy. Applicant agreed to make the reference of record 
in the present application by filing a Supplemental Information Disclosure Statement. However, the 
Examiner failed to provide a statement, as required by MPEP §707.05, 706.02(j), and 37 C.F.R. 
§ 1.106(b), of: (1) the relevant teachings of Bronson relied upon, (2) the differences in each claim over 
Bronson, (3) the proposed modification of Bronson necessary to arrive at the subject matter of each 
claim, and (4) an explanation why one of ordinary skill in the art at the time the invention was made 
would have been motivated to make the proposed modification. Applicant is therefore unable to 
address any findings or arguments against patentability based on the Bronson reference. Applicant 
reserves the right to present and have the Examiner fully consider de novo Bronson and argument 
concerning its applicability after the Examiner provides the required statement. 

The paragraph numbers of the remarks below correspond to the paragraph numbers in the 
Office Action 
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1 . Applicant agrees to add the legend Prior Art to Figures 1-13. Applicant further agrees to add 
legend for all symbolically illustrated structures in Figures 14-16. Applicant will submit replacement 
drawings for Figures 1-16 after receipt of a Notice of Allowance. 

2. Claims 2-5, 7-18, 92, 93, 95-97, 99, 100, 107, 108, and 1 1 1 each depend directly or indirectly 
from independent claim 1, contain all the limitations of claim 1, and were withdrawn from 
consideration by either Examiner Weldon or Examiner Liang. As stated above, the Examiner has 
agreed that claim 1, as amended, overcomes the Baker and Bronson references. Applicant requests 
that the Examiner reinstate these dependent claims and allow all claims depending from claim 1. 

Claim 1, as amended, is a generic claim directed to an apparatus for selecting a menu option, 
the apparatus including selectable regions outside the display screen. Species claims covered by 
claim 1 include independent claims 33, 39, 59, 70, and 78. Since the Examiner has agreed that claim 
1 as amended overcomes the Bronson and Baker references, Applicant requests that the Examiner 
reinstate and allow the independent species claims and all claims depending therefrom. 

3. In claims 1, 85, 94 and 106 the Examiner objects to the phrase "the particular selectable region 
intersected by a plurality of the successive locations" as indefinite. The Examiner indicates that 
appending the phrase "indicated by the movement related signal" will correct the problem. However, 
the successive locations, for example in clause (d) of claim 1, is definite and has its antecedent in 
clause (c), which reads "movement related signal receiving means for receiving a movement related 
signal indicating successive locations with respect to the display screen". Thus, as each of these 
claims stands, the successive locations are those indicated by the movement related signal. The 
proposed change is unnecessary. 

In claim 73 the Examiner objects to the phrase "at least one previously selected sequence" as 
indefinite. Claim 73 is directed to spelling a word by successively appending sequences of one or 
more letters. In this context, the phrase "at least one previously selected sequence" is clear. The 
phrase is used only once in the claim. Therefore there is no reference to "the at least one previously 
selected sequence". 

New claim 164, like claim 73, is directed to completing the spelling of a word, then speaking it. 
Claim 164 avoids any possible problems with the phrase "previously selected sequence" since this 
phrase does not appear in claim 164. 

In claims 80, 85, and 94 the Examiner objects to the phrase "the durations" since the durations 
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have not been defined. The phrase also appears in claims 107 and 113. "The duration of a period of 
intersection" is clear without further definition. Each period of intersection has a unique duration. 
Therefore, the phrase "the duration" is not indefinite and requires no antecedent. 

In claims 73, 80,91,94, 98, 101, 106, 114, 134, 147, 155, and 159-163 the Examiner objects to 
the phrase "a sequence of one or more ..." as confusing. Within the art, the phrase "a sequence of one 
or more letters" is understood to encompass one or more letters; it is not confusing. Furthermore, the 
phrase "a sequence of one or more ..." is well established claim language. The phrase appears in 
claims in many U.S. patents. Examples of recently issued patents including this phrase in their claims 
are: 

a. U.S. Patent No. 5,629,687 claiming "a sequence of one or more local commands"; 

b. U.S. Patent No. 5,617,570 claiming "a sequence of one or more remote procedure call 
requests"; 

c. U.S. Patent No. 5,608,869 claiming "a sequence of one or more parts "; 

d. U.S. Patent No. 5,600,724 claiming "a sequence of one or more symbols"; and 

e. U.S. Patent No. 5,598,560 claiming "a sequence of one or more elements". 

In claim 80 the Examiner objects to the phrase "moving at least part of the cursor" as confusing. 
Only part of the cursor might be moved, for example, if the cursor is positioned at the edge of the 
display screen, so that only part of the cursor is visible. 

4-9. In these Remarks, references are referred to as follows: 

Reference Cited herein as 

U.S. Patent No. 4,586,035 to Baker et. al. Baker 
U.S. Patent No. 4,305,435 to Bronson Bronson 
U.S. Patent No. 4,591,841 to Gunderson et al. Gunderson 
U.S. Patent No. 4,93 1 ,783 to Atkinson Atkinson 
Callahan, Jack; Hopkins, Don; Weiser, Mark; Shneiderman, Ben; Callahan 
"An Empirical Comparison of Pie vs. Linear Menus" in 
Human Factors In Computing Systems: CHI r 88, 
Conference Proceedings, 1988. 

Golding, V.G; Heneghan, M. J.; "Audio Response Terminal," Golding 

IBM Technical Disclosure Bulletin, vol. 26, no. 10B, March 1984. 

Lazzaro, Joseph J., "Computers for the Disabled," Byte, June, 1 993. Lazzaro 
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Collectively the above references are referred to in these Remarks as the cited references or 
cited art. 

The Office Action cited Baker as the primary reference against independent claim 1, its 
dependent claims 6, 90, 91, 98, 109, and 1 10, independent claim 94, independent claim 106, its 
dependent claim 1 13, independent claim 1 14, and its dependent claims 1 15-133. Bronson was not 
cited in the Office Action. As noted above, the Examiner agreed that claims 1, 94, 106, and 1 14 as 
amended above each overcome the Baker and Bronson references. Therefore, Applicant will not 
address the 35 U.S.C. §103 rejection, relying on Baker, of these independent and dependent claims in 
the Office Action . However, Applicant reserves the right to present argument and evidence and have 
them fully considered de novo should any of these claims be rejected on other grounds. 

Claims 1, 6, 90, 91, 94, 98, 106, 109, 1 10, 1 13, and 1 14-133 are believed patentable. 

All independent claims, with the exception of claim 1, are directed to voice output. Individuals 
who use voice output systems usually do so because their normal speech is impaired. However, 
impaired speech is generally a symptom of a systemic disorder, e.g. cerebral palsy, which affects the 
individual's motor control. Speech is affected because normal motor control is required to produce 
intelligible speech. Frequently, the systemic disorder also impairs the individual's ability to 
physically control a machine such as a voice output system through its normal interface, a keyboard 
or a mouse, interfaces which require relatively good fine motor control and/or precise pointing skills. 

Applicant observed individuals with neuromuscular disorders ("NMD operators"), in 
particular, cerebral palsy, during computer access . Many NMD operators, especially those with more 
severe impairments, cannot effectively use the conventional point and click or point and dwell (on- 
screen keyboard) computer interface (Specification, page 6, lines 5-9) because they cannot point 
precisely. Applicant isolated several constituent motor problems that contribute to imprecise 
pointing, for example, overshoot, tremor, drift, and involuntary movement accompanying voluntary 
movement (Specification, page 2, line 27 - page 3, line 6, and page 6, lines 16-18). Moreover, 
Applicant observed that the directional control exhibited by these individuals is often relatively good. 
The Applicant's discovery of the source of the problem of imprecise pointing is nonobvious. 
Consequently, applicant's claimed method and apparatus for solving this problem satisfies the 
nonobvious requirement of 35 U.S.C. §103. 

The invention enables NMD operators to control a voice output system and thereby verbally 
interact with their parents, teachers, fellow students, co-workers, and medical personnel. The 
invention may be used by literate users through an orthographic interface, or by illiterate users 
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through a symbolic interface using, for example, pictographs such as those shown in Figure 1 1 of the 
Application. 

Independent claim 73 stands rejected under 35 U.S.C. § 103 as being unpatentable over 
Callahan in view of Golding and Lazzaro. 

Independent claim 73 is directed to a voice output system for a user having impaired speech. 
The voice output system includes a display on which may be displayed a plurality of selectable 
regions. The selectable regions are visible and are located on the display within a polygon. Each 
selectable region is adjacent a side of the polygon such that together the selectable regions at least 
partially circumscribe a region on the display. Each selectable region is associated with a sequence of 
one or more letters. In addition to these elements, the voice output system comprises a voice output 
device and control means. The control means has three functions: (a) moving a cursor within the 
polygon responsive to a movement related signal; (b) appending selected sequences of letters together 
responsive to a succession of selection events; and (c) speaking the word spelled by the appended 
sequences. 

The voice output system of claim 73 differs from the combination of cited references 
structurally, functionally, and theoretically. Callahan, and in particular, the pie menu pictured in 
Figure 2 (Callahan, p. 95) and relied on in the Office Action (p. 10), is opaque, as are all pie menus 
disclosed in Callahan. Callahan states "[a] pie menu is a system facility for pop-up menus" 
(Callahan, p. 96, col. 1, line 19). Pop-up menu are opaque. Consequently, items underneath the pie 
menu are obscured when the pie menu is displayed. Callahan acknowledges this drawback, "pie 
menus consume greater screen area and become polynomially larger than linear menus in both height 
and width with increased item size and number of items" (Callahan, p. 96, col. 2, lines 6-9), and 
"there remains the problem of increased screen real estate usage. In one trial a subject complained 
because the pie menu obscured his view of the target prompt message." (Callahan, page 100, col. 1, 
lines 9-12). Thus, Callahan lacks the present invention's working region, referred to in claim 147 as 
the at least partially circumscribed a region on the display. 

The circle shown at the center of Callahan's Figure 2 is not a working region or application 
area. It is a dead zone within the pie menu and is the starting position of the cursor when the pie 
menu is displayed (Callahan, p. 96, col. 1, lines 47-49). The short distance from the center of the 
display to a target menu item is one of the defining characteristics and major advantages of pie 
menus, resulting in its superior target seek time versus linear menus (Callahan, p. 98, col 2., lines 34- 
35). This small distance is deliberate; it facilitates selection of a menu option from a pie menu 
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because the distance to any activation region is small. (Callahan, page 96, col. 2, lines 17-25). In 
Callahan's experiments this distance was 10 pixels. (Callahan, p. 99, col. 1, lines 50-51) 

By contrast, one of the objects of the present invention is to simultaneously display an 
application program window and a computer access menu which does not obstruct the application 
program window. (Specification, p. 1 1, lines 15-16). 

An important difference in results follows from having a relatively short average distance 
between the starting cursor location and a selectable region, as in Callahan, versus the relatively long 
average distance in the present invention. Within limits, as the distance to a selectable region 
increases, the time required to position a pointer on that selectable region increases, in accord with 
Fitts's Law. Fitts, P.M., "The information capacity of the human motor system in controlling the 
amplitude of movement, Journal of Experimental Psychology, 1954, vol. 47, pp. 381-391. Fitts 
hypothesized that a given human movement has a characteristic index of difficulty, called ID, where 

ID = log 2 (2A / W) 
where 

A = average amplitude of a human movement 
W = target width 

Within limits, movement time is a function of ID, according to the equation: 
MT-a + bID 
where 

MT = movement time 

a= constant 

b= constant 

ID - index of difficulty 

Investigators, with minor exceptions, have reported findings that provide general confirmation 
of Fitts's theory, (see citations reported in Fitts, P.M. and Peterson, J.R., "Information capacity of 
discrete motor responses", Journal of Experimental Psychology, vol. 67, pp. 103-1 12, 1964, at page 
104, col. 1, line 41 - col. 2, line 2). 

Thus, the mean movement time MT of an apparatus having selectable regions close to the 
starting cursor position differs from the mean movement time of an apparatus having selectable 
regions far from the starting cursor position. Pie menus and the present invention each have a 
different and characteristic index of difficulty ID. All other factors being equal, the index of 
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difficulty of the present invention is substantially greater than that of pie menus because the present 
invention requires a substantially larger movement amplitude, A. 

In accord with Fitts's Law, mean movement time is shortest in pie menus and longer for the 
present invention, all other factors being equal. 

Indeed, the authors of the Callahan reference explain the results of their investigation of pie 
and linear menus in terms of Fitts's Law. "An analysis of seek time based on Fitts's Law ... helps 
explain our results because the ratio of the distance (D) to target size (S) is smaller for pie menus. 
The fixed target distance and increased size of targets for pie menus decreases the mean positioning 
time as compared with linear menus. 11 (Callahan, page 99, col. 1, lines 1-11.) Thus Callahan teaches 
against increasing A, the average amplitude of human movement. Adding a working region in the 
center of a pie menu has just this result. 

Callahan provides no motivation to realize menu items as words. Words, and particularly 
words for voice output, are not among the menu items disclosed in Callahan. Callahan discloses the 
following menu items: compass directions, time, angular degrees, diametrically opposed or 
orthogonal function names, numbers, letters, ordinals, functions Login, Logout, Open, Close, 
Hardcopy, Shrink, Shape, Move, File, Save, Tie, Insert, Past and Cut (p. 96, col. 2, lines 1-2; p. 97, 
col. 1, lines 5-8; p. 95, Figure 1, p. 100, Figure 6). Callahan does not disclose or suggest menu items 
as words per se. In the present claims, words are themselves the menu items, not a representation of a 
function as in Callahan's Figure 1 . In the present claims, the words themselves are spoken. 

Callahan teaches against including menu items of letters in a pie menu. "Lists, like number, 
letters and ordinals are best suited for linear menus" (Callahan, p. 97, col. 1, lines 7-8). 

The proposed combination of Callahan, Golding and Lazzaro does not disclose or suggest 
spelling by appending successive selected sequences of letters, as claimed in claim 73. 

Claim 73 is believed patentable. 

Claim 104, depending from claim 73, limits claim 73 by adding the following elements: 

1 . the movement related signal is responsive to the head movement of the user; 

2. each of the plurality of selectable regions is adjacent an edge of the display; 

3. each of the succession of selection events includes a plurality of periods of intersection of the 
cursor and the intersected selectable region, each of the plurality of periods of intersection 
having a total duration equalling or exceeding a predetermined period; and 

4. the selection means further includes means for indicating the difference between the 
predetermined period and the total duration of the plurality of periods. 



18 



None of the cited art disclose or suggest (a) control means responsive to a plurality of dwell 
periods, or (b) means for indicating the difference between the predetermined period and the total 
duration of the plurality of periods. The practical and new results of these features are described 
below in connection with (a) claim 155, and (b) claim 134, respectively. 

Claim 104 is considered patentable independent of its parent claim. 

Claim 112, depending from claim 73, specifies that the control means moves the cursor only 
with the polygon. 

The combination put forth in the Office Action does not contain a confining polygon, either on 
the display or elsewhere. If the Examiner believes that a confining polygon is inherent in the applied 
art or in the proposed modifications to that art, the Examiner must so state so Applicant can respond 
to the assertion. 

The apparatus of claim 1 12 provides a critical new result, a result not attained by the methods 
and apparatuses disclosed in any of the cited references. As stated previously, because of their 
tendency to overshoot, many NMD operators cannot reliably or efficiently use on-screen keyboards. 
By confining the cursor to the polygon and locating the selectable regions within the polygon adjacent 
to its side, these operators can overshoot their intended selectable region and still select it. This result 
flows from the synergy between the polygon, the control means, and the location of the selectable 
regions. 

Claim 1 12 is believed patentable independent of its parent claim. 

A confiner, confining polygon or confining step is an element of claims 73, 80, 1 12, 1 14, 120, 
121, 146, 147, and 155. 

Independent claim 80 is directed to a method of speaking using a voice output system. The 
system includes a display displaying a plurality of selectable regions within a polygon. Selectable 
regions are located adjacent the sides of the polygon and at least partially circumscribe a region on the 
display. One or more of the selectable regions is associated respectively with a sequence of one or 
more characters. A cursor is moved only within the polygon responsive to a movement related signal. 
The user selects a sequence of characters by dwelling on the associated selectable region. The 
selected sequences of characters are appended and the word spelled by the appended sequences is 
spoken by a voice output device. 

Many of the differences between claim 80 and the applied art have already been discussed in 
connection with claim 73 and 1 12. However, claim 80, a method claim, introduces an additional 
factor which must be considered: the user's motivation for employing this method. Users of voice 
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output systems generally have a disorder affecting their motor control. One symptom of the disorder 
is impaired speech. Another symptom is impaired ability to voluntarily stop motion, resulting in a 
tendency to overshoot on-screen targets. Others symptoms may include reduced ability to prevent the 
movement of parts of the body, including the head, limbs and digits, muscle stiffness, weakness, 
limited range of motion, abnormal posture, involuntary muscle tremors, involuntary muscle activity 
causing involuntary motion, impaired ability to voluntarily stop motion, impaired ability to coordinate 
muscle activity, and/or impaired ability to sense the position of a part of the body. Any one of these 
symptoms may impair an affected individual's fine motor control. Moreover, while some individuals 
affected by a neuromuscular disorder may be able to exercise fine motor control with enormous 
effort, the struggle to do so often fatigues the individual, limiting the period of time the individual is 
capable or comfortable performing the fine motor control task (Specification, p.2, lines 5-13). Due to 
these symptoms, and particularly impaired ability to voluntarily stop motion, individuals with 
neuromuscular disorders tend to overshoot on-screen target areas. The combination of displaying the 
selectable regions within a polygon and adjacent its side, and moving at least part of a cursor only 
within the polygon enables some of these individuals to select targets they otherwise could not, or to 
do so more quickly or with less effort and concomitant fatigue. 
Claim 80 is considered patentable. 

Independent claim 163 explicitly claims the capability of allowing the user to select an 
overshot selectable region. The problem of overshoot is described in the extensively in the 
Specification: p. 2, line 33 - p. 3, line 3; p. 13, lines 28-30; p. 14, lines 10-18; p. 31, lines 6-10; p. 46, 
lines 1-14; and p. 62, lines 19-29. 

Claim 163 is directed to a method of speaking. The method includes three steps. One step is 
simultaneously displaying selectable regions adjacent a working region on a display. One or more of 
the selectable regions are associated respectively with a sequence of one or more characters, a 
sequence of one or more words, or a sequence of one or more symbols representing a sequence of one 
or more words. Another step is receiving a movement related signal indicating a location with respect 
to the display. The movement related signal is responsive to user movement indicating a potential 
user selection. Yet another step is speaking the sequence of one or more characters or words 
associated with a particular selectable regions responsive to a period of intersection of the particular 
selectable region and the location indicated by the movement related signal or the location on the 
display closest thereto, the period equalling or exceeding a predetermined period. The method allows 
the user may make a selection although the user movement overshoots the particular selectable region 
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on the display. 

Independent claim 85 is directed to multimodal selection from a plurality of menus. The 
apparatus includes a plurality of pluralities of menu options, i.e. a plurality of menus. A user 
indicates which one of the menus he desires. A sensor signal receiving means detects the user's 
indication and associates respectively the menu options of the selected menu with selectable regions. 
The user selects the desired menu option by dwelling on the associated selectable region. 

None of the cited references disclose or suggest multimodal selection from a plurality of 
menus, and none disclose the combination of sensor signal receiving means and a pointer. 

A multimodal menu interface speeds user interaction for individuals with impaired motor 
capability. If the user is able, he can select a menu with a first mode and, close in time, select from 
the selected menu by dwell. Multimodal interfacing is faster than single mode interfacing. In single 
mode selection by dwell requires dwelling for at least two selection thresholds, one after the other. 
Multimode interfacing, in contrast, may require dwelling for only one selection threshold. In 
addition, the selection of the menu may overlap in time with the selection of the menu option from the 
selected menu. Since the speed of the interface is usually the limiting factor in man-machine 
interactions for individuals with impaired motor capability, the improved efficiency of the interface 
results in greater productivity. Thus, there is a synergy between the elements of the combination of 
claim 85 not present in any of the cited art. 

Claim 86, depending from claim 85, comprises indicating means for indicating which plurality 
of menu options is associated with the selectable regions. This feature is not disclosed in any of the 
cited references. 

Claim 101, depending from claim 86, is directed to voice output. In claim 101, the selected 
menu option represents a sequence of one or more words. The apparatus further comprises a voice 
output device for speaking the sequence of one or more words responsive to the selection. The cited 
art does not disclose or suggest the combination of these elements with either sensor signal receiving 
means or selectable regions at least partially circumscribing a region on the display. The 
combination claimed in 101 facilitates man-machine interaction for individuals with impaired motor 
capability, as described in connection with claims 1 and 73. 

Claims 85, 86, and 101 are considered patentable. 

Claims 134-146 stand rejected under 35 U.S.C. §103. The Examiner relied primarily on Choi 
for this rejection. These claims and the reasons they are patentable are discussed below. 

Independent claim 134 is directed to a voice output system. The system includes a surface 
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including a selectable region associated with a first sequence of one or more characters, a sequence of 
one or more words, or a sequence of one or more symbols representing a sequence of one or more 
words. A movement related signal indicates a first location intersecting the selectable region and, at a 
later time, a second location intersecting the selectable region. The system also includes an indicator 
for indicating in a first manner at least the difference between the time the second location occurs and 
the time the first location occurs. The system speaks the sequence of one or more characters and/or 
words associated with the selectable region responsive to a first quantity that is a function of the 
difference equalling or exceeding a predetermined quantity. 

None of the cited references disclose or suggest an indicator for indicating the difference 
between the time the second location occurs and the time the first location occurs. 

The Examiner states that "Choi discloses a device comprising an indicator (level meter inside 
the sub-screen as shown in Fig. 3) which shows the remaining time before an activation signal 
(selection) is optionally applied." (Office Action, p. 9). 

There are several problems with extracting this particular element from Choi. The first is that 
the Choi reference is not pertinent prior art under 35 U.S.C. § 103. Choi would not have been known 
to a person having ordinary skill in the art to which the present Application pertains. Choi does not 
lie within the field of endeavor of the present Application. Choi relates to "a display apparatus for 
informing a user of a previously programmed recording in which, if it is time to record, a 
programmed recording mode is displayed on a television (TV) screen in a sub-screen, and more 
particularly to, a display apparatus for informing a user of a programmed recording in which a 
relevant message is generated through a PIP (picture in picture) circuit. The message can be 
generated during both a regeneration mode of a VCR (Video Cassette Recorder) or a TV broadcasting 
mode." (Choi, col. 1, lines 7-16). A person of ordinary skill in the art of the present Application 
would have no reason to consult the art of VCR programming. 

Choi addresses a problem not even remotely related to the present Application. As stated in 
Choi, "Generally, when a viewer wants to record a predetermined program of the TV with the VCR, 
the viewer inputs a recording time, a broadcasting channel and a broadcasting program in advance. 
Then, when it is the programmed recording time corresponding to the current time, the programmed 
broadcasting channel is automatically selected and the VCR is subjected to the recording mode, so 
that the input broadcasting program is automatically recorded. Therefore, at the programmed 
recording time, the programmed broadcasting program to record is unconditionally recorded even 
when the viewer is currently watching the TV or watching a regenerated program of the VCR. 
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"Accordingly, in the case where the viewer does not want to carry out the programmed 
recording, but wants to continue watching, since the recording mode of the VCR has to be converted 
to the original mode manually by the viewer, the viewer can not continue his watching without any 
interruption." (Choi, col. 1, lines 18-36). 

The problem addressed by Choi is not reasonably pertinent to the problem addressed by the 
present Application, and in particular the problem addressed by independent claim 134: producing an 
indication of progress toward or away from selection of a menu option. 

Assuming arguendo that Choi is pertinent prior art, Choi's teaching is very different from the 
indicator of claim 134. In Choi, the level meter is not affected by the viewer. The viewer cannot 
increase or decrease the level shown by the level meter. In claim 134, the indicator is responsive, not 
to a preprogrammed time, but to the difference between the times of a first and second location. Both 
locations are indicated by the user. Thus user movement affects the indicator of claim 134. 

Choi's invention is not directed to menu selection. In Choi, the level meter is not indicative of 
time to selection, but of time to the beginning of recording. The viewer is free to ignore level meter. 
It's purpose is just to notify viewer that he can override recording. If the viewer does nothing, 
recording occurs as previously programmed. In contrast, claim 134 is directed to menu selection. If 
there is no user interaction, then there is no intersection of a location and a selectable region, and no 
selection. 

Claim 134 is believed patentable. 

In claim 135, which depends from claim 134, the indication of the difference includes any one 
of a visible signal, an audible signal, and a tactile signal. Claim 135 is deemed patentable. 

In claim 136, which depends from claim 134, the indicator may indicate a second quantity 
which is a function of the difference between (a) a predetermined period, and (b) the difference 
between the time the second location occurs and the time the first location occurs. In the preferred 
embodiment, the predetermined period is the selection threshold. Thus the indicator indicates the 
time remaining until selection. None of the cited references disclose or suggest such an indicator. 
Claim 136 is believed patentable independent of its parent claim. 

Claim 137 depends from claim 134. Claim 134 further includes a third location, occurring at a 
time after the time of the second location. The third location does not intersect the selectable region. 
The indicator is further operative to indicate the non-intersection. None of the cited references 
disclose or suggest such an indicator. Claim 137 is believed patentable independent of its parent 
claim. 
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Claim 138 depends from claim 134. Claim 134 includes a third location like that of claim 137, 
and also a fourth location. Thus the apparatus of claim 138 indicates a period of non-intersection 
following a period of intersection. None of the cited art discloses or suggests such an indicator. 
Claim 138 is considered patentable independent of its parent claim. 

Claim 139 depends from claim 138. In claim 139, the indicator is further operative to produce 
an output signal which varies in at least one way as the difference between the time the second 
location occurs and the time the first location occurs increases and varies in at least the opposite way 
as the difference between the time the fourth location occurs and the time the third location occurs 
increases. Thus the period of intersection and the period of non- intersection are indicated in opposite 
ways. None of the cited art discloses or suggests such an indicator. Claim 139 is considered 
patentable independent of its parent claim. 

Claim 140, which depends from claim 134, specifies that the indication includes a 
modification in brightness. None of the cited art discloses or suggests such an indicator. Claim 140 
is considered patentable independent of its parent claim. 

Claim 141 depends from claim 134. In claim 141, the indicator is further operative to indicate 
in a second manner the first quantity equalling or exceeding the predetermined quantity, i.e. the event 
to which the voice output device is responsive. Claim 141 is considered patentable. 

In amended claim 142, depending from claim 141, is directed to relative indications. The 
indicator indicates in a first manner the difference between the first and second locations (claim 134), 
and indicates in a second manner the first quantity equalling or exceeding the predetermined quantity 
(claim 141). In amended claim 142, the indication in the first manner includes a slight modification 
and the indication in the second manner includes a marked modification. None of the cited art 
discloses or suggests differing relative indications. Claim 142 is believed patentable independent of 
its parent claim. 

Claim 143, depending from claim 141, specifies that the indication in the second manner 
includes a modification in hue. None of the cited references teach or suggest such an indicator. 
Claim 143 is considered patentable independent of its parent claim. 

Claim 144 depends from claim 134. In claim 144, the indicator intersects the selectable 
region. None of the cited references show the claimed structure. Claim 144 is considered patentable 
independent of its parent claim. 

Claim 145 depends from claim 144 and further specifies that the selectable region and the 
indicator are coterminous. None of the cite references show the claimed structure. Claim 145 is 
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considered patentable independent of its parent claim. 

Claim 146 depends from claim 134. In claim 146, the voice output system further includes a 
confiner for preventing the movement related signal indicating the first and second locations from 
moving beyond a side of the selectable region. None of the cite references teach such a confiner. 
Claim 146 is considered patentable independent of its parent claim. 

Claims 147-154 stand rejected under 35 U.S.C. §103. For this rejection, the Examiner relied 
primarily on Baker. Independent claim 147 claims a structure very similar to that of claim 1 14. The 
Examiner has agreed, as noted above, that amended claim 1 14 overcomes Baker and Bronson. 
Presumably, if similarly amended, so would claim 147. Applicant is willing to so amend claim 147 if 
the Examiner agrees that so amended the claim is patentable. 

The Examiner cited Atkinson for the association of a menu option with a plurality of 
submenus. There are several problems with extracting this particular element from Atkinson. The 
first is that the Atkinson reference is not pertinent prior art under 35 U.S.C. §103. A person of 
ordinary skill in the art of the present invention would not consult Atkinson because Atkinson is not 
concerned with a problem shared with the present invention. Atkinson is directed to a problem 
associated with pull-down menus. "[S]ince the pull-down window generates a menu window which 
is typically in a fixed position, any images near that fixed position are frequently obscured" 
(Atkinson, col. 2, lines 2-5). In claim 147 selectable regions associated with both menu options and 
submenu options are located adjacent the periphery of the working region. An example of such a 
structure is depicted in Figures 1 7 and 18 of the present application. Images in the working region are 
not obscured, the problem Atkinson addresses. The voice output system of claim 147 simply does not 
have this problem. 

The Examiner considers that it is well known in the art that a menu option may be associated 
with a plurality of submenus, and therefore is not novel. (Office Action, p. 8). Whether a menu 
hierarchy is novel or not is irrelevant under 35 U.S.C. §103 which requires analysis of obviousness of 
a claim as a whole, not novelty of an element thereof. 

In the Office Action, the Examiner concluded that claim 147 was obvious in view of a 
proposed a combination of a modification of Baker with selected elements of Gunderson, Callahan, 
Golding, and Atkinson. However, Baker teaches virtual menu items, i.e. not displayed during normal 
operations. This is critical to achieving Baker's intended result. Atkinson teaches displaying all 
menu items, i.e. both the command options in the menu bar and the command items in the pull-down 
menu. Displaying menu items is critical to Atkinson. Otherwise the user must memorize a complex 



25 



menu hierarchy. Indeed Atkinson's invention is directed to keeping a submenu continuously visible at 
a desired location on the display screen. These references are thus in conflict and not combinable. 

The Examiner is apparently using the present Specification as a guide through the prior art for 
reconstructing the claimed invention. This is evident from the Examiners particular modifications of 
selected elements of the cited references. For example, Atkinson is exclusively directed to linear 
menus. (Atkinson, Figures lb, 2, 3a, and 3b, col. 1, lines 42-52, col. 2, lines 29-34, col. 2, lines 67 - 
col. 3, line 2, col. 3, lines 66-68, col. 4, line 63, col. 5, lines 60-65, col. 6, line 28). Nothing in 
Atkinson discloses or suggests a perimeter menu as claimed. Therefore, assuming arguendo that 
Atkinson can properly be combined with the proposed combination of Baker selected elements of 
Gunderson, Callahan, and Golding, the most natural combination would be a menu window 
containing a linear menu of command items, displayed adjacent a displayed menu item. This 
combination lacks the functionality of the present invention. Individuals with impaired motor control 
have great difficulty selecting from linear menus, and thus would not be able to use such a menu 
window. 

An alternative combination of Baker and Atkinson, eschewed by the Examiner, is to make 
Atkinson's menus virtual, i.e. not displayed in normal operation, as taught by Baker. The 
impractical ity of this combination is immediately apparent and illustrates the inappropriateness of 
combining these references. The user must memorize a complex menu hierarchy. Suppose he 
remembers the location of the Edit command option. Once he selects Edit, he must remember where 
the Undo, or Cut, or Copy command item resides in the next level of the virtual menu hierarchy. 
Alternatively, the user can position the cursor over successive virtual menu items, displaying them, 
until he finds the one he wants. The reason Baker disclosed only one level for his virtual menu is thus 
apparent: a virtual menu places a memorization burden on the user. Baker recognized this drawback 
and proposed a training template. (Baker, col. 5, lines 33-34). The present invention overcomes this 
drawback in Baker by doing what Baker teaches against: displaying menu options during normal 
operations. 

Claim 148, depending from claim 147, defines a relationship within the menu hierarchy 
described in the parent claim. In claim 147 the sequence of words, characters, or symbols associated 
with the menu option includes or represents a class of characters or words associated with the 
submenu option. Claim 148 is believed patentable. 

Claim 149, depending from claim 148, further limits the class described in its parent claim to 
one of the listed species. None of the cited references disclose or suggest any of the classes claimed. 
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Claim 149 is believed patentable independent of its parent claim. 

Claim 150 depends from claim 148. In claim 150, responsive to the selection of the menu 
option, the individual characters, words, or symbols associated therewith are associated respectively 
with and simultaneously displayed by the second plurality of selectable regions for eventual selection 
by the user. Claim 150 is deemed patentable. 

Claim 151 depends from claim 148. Claim 151 addresses the distance on the display area 
between the specific selectable region and each of at least two of the second plurality of selectable 
regions. This distance is responsive to a relative frequency of use of the sequence of characters, 
words, or symbols associated with each of the at least two selectable regions. None of the cited art 
addresses the distance between a menu option associated with a submenu and the submenu options. 
Claim 1 5 1 is considered patentable independent of its parent claim. 

Amended claim 152 depends from claim 151. In claim 151, the sequence of characters, words, 
or symbols associated with one of the at least two selectable regions is more frequently used than the 
sequence of characters, words, or symbols associated with another one of the at least two selectable 
regions. The distance between the specific selectable region and the one of the at least two first 
selectable regions associated with the more frequently used sequence is less than the distance between 
the specific selectable region and the another one of the at least two first selectable regions. None of 
the cited art discloses or suggests this feature. Claim 152 is considered patentable independent of its 
parent claim. 

Claim 153 depends from claim 148. In claim 153 the position of at least two characters, 
words, or symbols within the displayed specific sequence indicates the position of the one of the 
second plurality of selectable regions on the display area associated with the character, word, or 
symbol. None of the cited references disclose or suggest this feature. Claim 153 is considered 
patentable independent of its parent claim. 

Claim 154, depending from claim 147, further limits the symbols described in its parent claim 
to one of the listed species. None of the cited references disclose or suggest the combination of claim 
154. Claim 154 is deemed patentable. 

Independent claim 155 is directed to a voice output system. The system includes a display 
area or surface including a working region with a periphery. Selectable regions are located adjacent 
the periphery of the working region. Each of the selectable regions has an external boundary which 
includes the side of the selectable region furthest from the working region. Additionally, each 
selectable region either has a confiner for preventing a movement related signal indicating a location 
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from moving beyond the external boundary of the selectable region or has an activation area 
extending beyond the external boundary of the selectable region and beyond the display area. Each 
selectable region is associated respectively with and simultaneously displays a sequence of one or 
more characters, a sequence of one or more words, or a sequence of one or more symbols 
representing the sequence of one or more words. Selection is responsive to a plurality of periods of 
intersection of the movement related signal and a particular selectable region or the activation area 
associated therewith. The selected sequence is spoken by a voice output device. The voice output 
system provides the user with the ability to select a particular selectable region while overshooting 
the particular selectable region. 

None of the cited references disclose either (a) a confiner or activation areas outside the 
display area, or (b) selection responsive to the duration of a plurality of periods of intersection. 

The purpose of Gunderson's invention is to allow "the user to locate a spot on the face of a 
video screen without having to physically touch the screen ..." (Gunderson, col. 2, lines 61-63, 
emphasis added). That Gunderson's invention deals exclusively with detecting a location on the video 
screen, not outside of it, is readily apparent from the specification. Gunderson's optical detector 
detects a plurality of raster lines of a CRT (Gunderson, col. 2, line 67 - col. 3, line 3) drawn on the 
face of the CRT (Gunderson, col. 4, line 65 - col. 5, line 2, col. 5, lines 14-15, 30-33, 41). The 
optical detector and the CRT's electronics are connected to a signal processor which determines the 
location of the point indicated by the optical detector (Gunderson, col. 3, lines 3-8 and 44-48, col. 5, 
lines 33-38). Since no raster is drawn outside the video screen, this apparatus cannot detect when the 
optical detector is pointing outside the video screen. Gunderson does not disclose or suggest a 
confiner. Indeed, Gunderson states that "[i]f no detections are made or averaged in a frame, a special 
code message is put out ... to indicate ... that the pointer 1 1 is not directed at the screen 16." 
(Gunderson, col. 12, lines 35-40. See also col. 16, lines 24-27). 

Furthermore, even with Gunderson's optical detector pointed at a location on the video screen, 
Gunderson's invention is responsive to dwell "for a predetermined period of time" (Gunderson, col. 5, 
lines 64-65), not a plurality of periods. Individuals with impaired motor control often exhibit an 
inability to maintain a location indicated by a pointer steadily at or near a desired location. Tremor 
and/or drift may pull the location indicated by the pointer off the intended target before Gunderson's 
single dwell period is satisfied or cause the location indicated by the pointer to oscillate across the 
edge of a target region. Gunderson's user must dwell again for the full predetermined period, perhaps 
repeatedly. In contrast, selection responsive to a plurality of periods of intersection may allow that 



28 



same user to make his intended selection in spite of tremor and/or drift. On return to a selectable 
region, the user need only dwell for a period equal to the difference between the selection threshold 
and the previously accumulated dwell time for that selectable region 
Claim 155 is believed patentable. 

Claims 156 and 157 include a period of non-intersection of the movement related signal and 
the particular selectable region which affects when the particular selectable region is selected. In 
claim 155 selection is responsive to a quantity equalling or exceeding a predetermined quantity. The 
quantity is a function of the duration of a plurality of periods of intersection of the movement related 
signal and the particular selectable region or the activation area associated therewith. In claim 156, 
the quantity is further a function of a duration of non-intersection of the movement related signal and 
the particular selectable region or the activation area associated therewith. In claim 157, the quantity 
which must equal or exceed the predetermined quantity, varies one way as the duration of one of the 
periods of intersection increases and varies in an opposite way as the duration of the period of non- 
intersection increases. None of the cited references discloses or suggests selection responsive to a 
period of non-intersection. Claims 156 and 157 are each considered patentable independent of their 
respective parent claims. 

Independent claim 159 is directed to a voice output system. The voice output system includes 
a display area including a working region with a periphery. It also includes a display device for 
displaying menu options on the display area. Each menu option is displayed adjacent the periphery of 
the working region, each menu option associated respectively with a user activatable switch outside 
the display area. The switch is positionable with respect to the location of each menu option for 
selection thereof. Each menu option is associated respectively with a sequence of one or more 
characters, a sequence of one or more words, or a sequence of one or more symbols representing a 
sequence of one or more words, for selection via the switch. The voice output system also includes a 
voice output device for speaking the sequence of one or more characters or words associated with a 
particular menu option, in response to the position of the switch corresponding to the particular menu 
option for a period equalling or exceeding a predetermined time period. 

The Examiner states that "[a]s to claims 159, 160, 162, the menu items 43-50 in Fig. 8 of 
Baker reads on the a user activatable switch as claimed." (Office Action, p.7). However, claim 1 59 
requires a plurality of switches, since each menu option is respectively associated with a user 
activatable switch. The difference is important to users with impaired motor control and leads to a 
new and unexpected result. These users are unable to efficiently use a standard keyboard or mouse. 
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(Specification p.2, lines 32-33). Some of these users are able to use a joystick which may include a 
plurality of switches. The voice output system of claim 1 59 speaks the words associated with a 
particular menu option responsive to the user positioning the joystick corresponding to the particular 
menu option for a predetermined period. Some users with spasticity can position the joystick so the 
switch corresponds to the particular menu option for a predetermined period. Users with spasticity 
cannot point at a relatively small target, such as shown in Baker's Figure 8, and simultaneously close 
a switch. Consequently, such a user can operate the voice output system of claim 159 and cannot 
operate Baker's apparatus. 

Other differences between the voice output system of claim 159 and Baker are that in the voice 
output system the menu options are displayed in normal operation, the use of words as menu options, 
the inclusion a voice output device, and the function of speaking. Claim 159 is believed patentable. 

Independent claim 160, in addition to the elements described in connection with claim 159, 
includes an indicator for indicating at least the difference between the time period during which the 
position of the switch corresponds to the particular menu option and the predetermined time period. 
Dwell indication was discussed above in connection with claim 134. Claim 160 is believed 
patentable independent of claim 159. 

Independent claim 161, in addition to the elements described in connection with claim 159, 
includes a menu hierarchy including a plurality of menu options. A specific one of the menu options 
is associated with a submenu comprising a plurality of submenu options. Each of the submenu 
options is associated respectively with a sequence of one or more characters, a sequence of one or 
more words, or a sequence of one or more symbols representing the sequence of one or more words. 
The voice output device speaks the particular sequence of one or more characters or words associated 
with a particular one of the submenu options, in response to the position of the switch corresponding 
to the specific menu option for a first time period equalling or exceeding a first predetermined time 
period and thereafter to the position of the switch corresponding to the particular menu option for a 
second time period equalling or exceeding a second predetermined time period. Selection of 
characters or words from a menu hierarchy was discussed above in connection with claim 147. Claim 
161 is believed patentable independent of claim 159. 

Independent claim 162, in addition to the elements described in connection with claim 159, 
requires selection responsive to a plurality of periods in which the position of the switch corresponds 
to the particular menu option. Selection responsive to a plurality of periods of intersection was 
discussed above in connection with claim 155. Claim 162 is believed patentable independent of claim 
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159. 



In view of the foregoing, and in summary, claims 1-18, 33-51, 59-60, 70, 73, 78, 80, 85, 86, 
90-101, 104, 106-157, and 159-169 are considered patentable over the cited references. Favorable 
reconsideration of the Application, as amended, is respectfully requested. 

Respectfully submitted, 



Donald K. Forest 
Applicant 
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